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Abstract 

We prove a Gaussian process approximation for the sequence of ran- 
dom compositions of a two-color randomly reinforced urn for both the 
cases with the equal and unequal reinforcement means. By using the 
Gaussian approximation, the law of the iterated logarithm and the func- 
tional limit central limit theorem in both the stable convergence sense 
and the almost-sure conditional convergence sense are established. Also 
as a consequence, we are able to to prove that the distribution of the urn 
composition has no points masses both when the reinforcement means are 
equal and unequal under the assumption of only finite (2-|-e)-th moments. 
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1 Introduction 

Asymptotic properties, including the strong consistency and asymptotic nor- 
mality, of urn models and their applications are widely studied in recent years 
under various assumptions concerning the updating rules, for example, one may 
refer to Chauvin, Ponyanne and Sahnoun (2009), Bai, Hu and Rosenberger 
(2002), Bai and Hu (2005), Janson (2004,2006), Zhang, Chueng and Hu (2006) 
etc. In this paper, we consider a kind of two-color urn model, called the ran- 
domly reinforced urn (RRU) model, which is a generalization of the original 
Polya urn (c.f., Eggenberger and Polya (1923), Polya (1931)). The main issue 
of this model different from most urn models in literature is that, as shown, 
the proportions of balls in the urn will not converge to a non-extreme constant 
and the numbers of different type balls may increase in different speeds. This 
issue makes its asymptotic properties quite different from the those of other urn 
models and difficult to study. 
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The RRU model is described as follows. Consider a two-color urn with the 
initial urn components Yq = (io,i) ^0,2)), where Yb.fc > is the number of type 
k balls. The urn is sampled sequentially. Suppose the urn components are 
Ym = (i^m,ii ^"1,2) after m samplings. At the (m + l)-th sampling, a ball of 
type k is drawn with a probability 

Y 

Pm+i,k = where \Yrn\ = ¥^,1 + ¥^,2- 

I I 

And the sampled ball is replaced in the urn together with a nonnegative random 
number Um+i,k of balls of the same type k, generated from a distribution Hk with 
mean nik > 0. This is the model introduced and formally named the randomly 
reinforced urn in Mulier, Paganoni and Secchi (2006a). But it would appear 
in earlier literatures in different versions. For example, Durham and Yu (1990) 
proposed a similar model for sequential sampling in clinical trails. In our RRU 
setting, the numbers of balls take positive real values, not necessary integers. 
When Um+1,1 = Um+1,2 = a is a constant and a positive integer, a RRU is 
the original Polya urn (c.f., Eggcnberger and Polya (1923), Polya (1931)) which 
is very popular in literatures. The RRU model is of fundamental importance 
in many areas of applications, for instance in economics (c.f., Erev and Roth 
(1998), Bcggs ( 2005), Hopkins and Posch (2005)), in information science (c.f., 
Martin and Ho ( 2002)), in resampling theory etc. In clinical trial studies, the 
RRU model is utilized to define a response-adaptive design focusing to reduce 
the expected number of patients receiving inferior treatments (c.f, Durham, 
Flournory, Li (1998), Li, Durham and Flournory (1996), Melifer, Panganoni 
and Secchi (2006a,b), Peganoni and Secchi (2007), May and Flournory (2009) 
etc). 

Suppose the reinforcement distributions /ii and 112 have bounded supports. 
In Melifer, Panganoni and Secchi (2006a), it is showed that the sequence {Z„ = 
F„,i/|l^i|} of the random sample proportions in the urn converges to almost 
surely to a random limit Z^o G [0,1]- When fii = IJL2, Crimaldi (2009) proved 
a central limit theorem by showing almost-sure conditional convergence to a 
Gaussian kernel of the sequence {^Jn{Zn — Zoo)}- Aletti, May and Secchi (2009) 
extended Crimaldi's result to a general case that reinforcement means mi and 
m2 are equal and proved that has no point masses in [0, 1] by using this kind 
of conditional central limit theorem. When the means mi and m2 are different, 
the limit proportion of a RRU is showed to be a point mass either 1 and 
by Bcggs (2005), Hopkins and Posch (2005) and Melifer, Panganoni and Secchi 
(2006a) under the assumption that the supports of /ii and /i2 are bounded from 
0, and by Aletti, May and Secchi (2009) only under the assumption that and 
^12 have bounded supports. May and Flournory (2009) proved that the sequence 
{Xn,i/¥^2^"^^} converges to almost surely to a random limit Voo € (0, 00) both 
when mi = m2 and mi ^ m2. 

The purpose of this paper is to establish the Gaussian process approximation 
of the sequence {Zn} when nii = 7712 as well as the sequence 
when mi 7^ TO2, under the assumption that /Ui and ^2 have only finite (2-t-e)-th 
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moments. This assumption is much weaker than that ni and /X2 have bounded 
supports. We will show that both these sequences can be approximated by a 
tail stochastic integral with respect to a Brownian motion mixed with a random 
variable. It is interesting that, as we will find, the mixed Gaussian process for 
approximating is nearly independent of the urn composition to be approximated. 
Our Gaussian process approximation enables us (i) to establish the law of the 
iterated logarithm; (ii) to establish the functional limit central limit theorem in 
both the stable convergence sense and the almost-sure conditional convergence 
sense; (iii) to prove that the limit ■i/'oo (resp. Zoo) has no point masses in [0, oo] 
(resp. in [0, 1]) when mi ^ m2 (resp. when mi — m2) under the assumption 
that /ii and /i2 have only finite (2 + e)-th moments. Another implication of 
our Gaussian approximation is that we are able to establish the central limit 
theorem in a simple way for the random number Nn^k of draws, where Nn^k is 
the number of type k balls being drawn in the first n samplings. In a response- 
adaptive design in clinical trials driven by a RRU model, Nn.k is the number of 
patients allocated to treatment fc, and its asymptotic behaviors are of particular 
interest. 

For the generalized Friedman urn models, Bai, Hu and Zhang (2002) and 
Zhang and Hu (2009) established the Gaussian approximation for both the urn 
proportions Yn^k/n and the sampling proportion Nn^k/n. But the RRU which 
we consider here is not covered by their assumptions. The main reason is that 
the mean replacement matrix diag{mi,m2) of a RRU is not irreducible and 
hence the limit of Yn^k/n and Nn^k/n is not a constant in (0, 1). 

The paper is organized as follows. The main approximation theorems with 
applications for equal and unequal reinforcement mean case are stated in Section 
[2] and Section [3l respectively, and the proofs of the approximations appear in 
the last section. Some remarks on unsolved problems are discussed in Section 

m 

In the sequel of this paper if having not been specially mentioned, {Ui,i, Ui,2), 
I — 1,2,... are assumed to be independent identically distributed random vec- 
tors with finite second moments. Let Xj^^k be the result of the m-th drawing, 
i.e., Xm,k = 1 if the m-th drawn ball is of type fc, and otherwise. It is obvious 
that Nm,k = Z^jli ^rn,k and Xm,i+Xm,2 = 1- Denote Tn = <j{Ui^k, Xi^k,Yik ■ 
k = 1,2; I = 1, . . . , n) be the history cr- field generated by all the observations 
up to stage n, and Too ~ Vn Further, for two positive sequences {a„} and 
{bn}, we write an = 0(fe„) if there is a constant C such that a„ < C6„, a„ ^ bn 
if a„/6„ ^ 1, and a„ sa 6„ if a„ = 0(6„) and 6„ = 0(a„). 



2 Equal reinforcement mean case 

In this section, we consider the case of mi = m2 > 0. Let tr^ = E[(J7i.fe/mfe)^], 
fc = l,2, 

Zn = ^ ^"'V , Zoo(w) = liniZ„(cj), H{uj) = ^ + ^l, ■ 
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To start, we shall assume P(Zoo = 0) = P{Zoo = 1) = 0, for otherwise H may 
have no definition. This resuh is proved by May and Flournoy (2009) under the 
condition that the reinforcement distributions and yLt2 have bounded supports. 
The next theorem tehs us that May and Flournoy's condition can be relaxed at 
least to the assumption of finite second moments. 

Theorem 2.1 Suppose mi = m2 > 0, H/f j. < oo for some q > 1, k = 1,2. 
Then the limit Zoo exists almost surely and P(0 < Z^a < 1) = 1. 

The following theorem is the main resuh on the Gaussian approximation. 

Theorem 2.2 Suppose mi = m2 > 0, H/fj, < oo, k = 1,2, where 2 < p < 
4. Then (possibly in an enlarged probability space) there is standard Brownian 
motion B{y) such that 



Furthermore, the Brownian motion B{y) can be constructed with a filtration of 
a-fields {^^n} and a non- decreasing sequence of stopping times {Tn} satisfying 
the following properties: 

Property (a) Tn C^n, Tn is ^n measurable; 

Property (b) r„ = nH + o{n'^/P) a.s.; 

Property (c) Conditional on Tn, B{Tn + y) — B[Tn), y > 0, is also a standard 
Brownian motion. 

Remark 2.1 Denote W{t) — — t y^^dB{y) . By checking the covariance 
function, it is easily seen that W(t),t > is also a standard Brownian motion. 

Remark 2.2 The process in h2.1\) for approximating is a tail stochastic integral 
respective to the Brownian motion. It looks like to be independent of Zn. Actu- 
ally, according to Property (b) nH can be replaced by Tn, and \/T^ Jj, y^^dB{y) 
is indeed a normal random variable which is independent of Z„ . We will illus- 
trate this interesting property in Corollary \2.3\ in more details. 

We win prove Theorem 12.21 by first approximating Z„ — Z^o to an infinite 
summation of a weighted martingale sequence and then approximating the mar- 
tingale to a Brownian motion by applying the Skorokhod embedding method. 
The detail proofs of Theorems 12.11 and 12.21 will be stated in Section [Xj In the 
sequel of this section, we give several corollaries as applications. Define 




(2.1) 



where 




ifp^2 

i/ 2 < p < 4. 
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a„(w) = ^Znil - Zn)^ (1 - Z,,)al + Z„ct|. 
The first corollary is the following law of the iterated logarithm. 
Corollary 2.1 Suppose mi — m2 > 0, H7^j, < oo, k = 1,2. Then 

V^(Z„ — Zryaj ~ 

limsup — — = G U.S.. 

ra-)-oo v21oglogn 



Proof. Write 7(x) = ^x/ (2 log log x), G(a;) = - y-^dB{y). By ([21]), we 
need to show that 

linisup7(ni?)G(niJ) = linisup7(T)|G'(r)| = 1 a.s. (2.2) 



Note that j{x)G{x) = xG{x) j \J2x log log x, and that a;G'(a;) is also a standard 
Brownian motion. (12.21) follows from the law of the iterated logarithm of the 
Brownian motion. □. 

The next corollary is on the functional cental limit theorem. 

Corollary 2.2 Suppose mi — m2 > 0, EUf < oo for some p > 2, k = 1,2. 
Define 

Wn{t) = tV^(Z[„t] - Zoo),t > 0. 

Then 

Wn{-)^GB'{-) Stably, (2.3) 

in the Skorokhod Topological space D(0,oo), where B'{t) is a standard Brownian 
motion which is independent of !Fao ■ In particular, 

hm P ( ^^^0<^<J^liZi-Z^) ^ ^ ^ ^ ^2.4) 



n—yoo 



Here the stable convergence in i2. 3\) means that for any bounded and (uniformly) 
continues function f : D{0, oo) — > (— oo, oo), 

E[f{Wni-))lE] ^ E[f{aB'{-))lE] for any event E. 



Remark 2.3 The convergece i2.4\ ) does not depend on the unknown value of 
Proof (HHl) is due to fact that 

l{Zi-Zn) Wnit)-tWnil) d 

max _ — -= sup ^ sup [B [t) - tB [l)) 

o<i<n a^n o<t<i o" o<t<i 

and that B' {t) - tB'{l) is a Brownian bridge. For (g^l), note that W{x) = 
—t y~^dB{y) is also a standard Brownian motion. By (12.11) . 

n(Z„ - Zoo) - -^W{nH) = o{n^/^) a.s., 
V H 
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which implies that for any T > 0, 



sup 

0<t<T 



Wn{t)-a 



W{nHt) 



'nH 



0(1) a.s.. 



For the Brownian motion W{-), we have 



W {n-) / \/n ^ B' {■) mixing, 

i.e., for any given event E with P{E) > 0, the conditional distribution of 
W (n-) / ^/n converges to a Brownian motion. It follows that 

{a/VH,H,W{n-)/y/^) 4 {a / y/W , H , B' {■)) stably. 

WjnH-} 
~iH 



is a continuous function of {(j/\/H,H,W{n-)/y/n) of the 



Note that a 
form f{r,h,x{-)) — rx{-h). It follows that 



.W{nH- 



d ^B'{-H) d ^ . , , 
(T = aB (•) stably. 



The proof is now completed. □ 



Corollary 12.21 implies the central limit theorem for •\/n(Z„ — Zoo). Aletti, 
May and Secchi (2009) proved a strong version of the central limit theorem. For 
every Borel set B, every uj, and n = 1, 2, . . . ,, define 

Kn{iO,B) = P{V^iZn - Zoo) e B\J^„){UJ), 

i.e., Kn is a version of the condition distribution of ^/n{Zn — Zoo) given 
Aletti, May and Secchi (2009) showed that, if mi = 7712 and the distributions 
of /ii and /i2 have bounded supports, then for almost every w, the sequence of 
probability distributions Kn(u!, ■) converges weakly to the normal distribution 

N{0,a\Lu)). 

We denote this kind of convergence by 

V^{Zn-Zoo)\^ ^ N{0,a^u;)) a.s.. (2.5) 

This kind of conditional central limit theorem was first established by Grimadi 
(2008) for a special case that — ^2- Aletti, May and Secchi (2009) also showed 
that (|2.5p implies that Zoo has no point masses in (0,1). Our next corollary 
tells that (|2.5p and a type of conditional functional central limit theorem are 
conclusions of the Gaussian approximation. 

Corollary 2.3 Suppose mi = m2 > 0, EUf f. < 00 for some p > 2. k = 1,2. 
Then ^ 

Vn{Z 00 - Z[nt]) -o-n\/T\i [ = o(n"^) a.s. (2.6) 



sup 

t>l 
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for some e > 0, and further, Bn{t) = —t^/T\l J^^y~^dB{y), t > 1, is also a 
standard Brownian on [l,oo) which is independent of Tn- 
As a consequence, 



Wn{-)\ 



a{uj)B'{-) a.s. 



(2.7) 



in the Skorokhod Topological space D[l,oo), where Wn{t) is defined as in Corol- 
lary B'{t) is a standard Brownian motion which is independent of J-ao- 

In particular, \2.5\) holds. Zoo has no point masses in (0, 1), and there exists 
a sequence of standard normal random variables for which C,n is independent of 
Tn and 

y/n{Zn - Zoo) = <?nCn + o(n~') O.S., (2.8) 



Proof. We first prove dH]). By (1^ . 



sup 

t>i 



Vn{Zryo - Z[nt]) - ^00(1 - Zoo)Hy/n 

J n 



dB{y) 



tH y 



o{n ^) a.s. 



Let H„ = al/Zn + a\l{\ - Z„). Note that Z„ - Z^o = 0{^/rF^og\ogn) a.s. 
by Corollary [Q and Tn/n = H + o{n'^/'P^'^) a.s.. It follows that ^/nZoo{l - 
Zoo)-ff-CT„V7^ = o(n2/p-i/2) a.s.. Further, 



t>i 



dBiy) 



ntH y 



n^/p^-L/^Oix/n-noglogn) = o{n-') 



It remains to show that 



sup 
t>i 



dBiy) 



tH y 



o{n 



-112- 



a.s. 



Write a„ — T„ — nH. Note that W{x) — —x y ^dB{y) is a standard Brow- 
nian motion and 



dB{y) W(nHt + a^t) - W{nHt) W{ntH) nH - T„ 



itH y Tnt ntH Tn 

The second term on the right hand of the above equality does not exceed 



0{y/n-Hog\ogn)o{n'^'P~^) 

uniformly in t > 1 almost surely, by the law of the iterated logarithm. The first 
term does not exceed 



^ / ^/ant{\og{nH) + loglog(niJt)) ^ i/p-i/, ^l/2^ 

O I — I = o{n 'P (logn) I ) 



uniformly in t > 1 almost surely, by the path properties of a Brownian motion 
(c.f. Hanson and Russo (1983)). The proof of p.6p is now proved. 
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Bn{t) - -<V^ / 

Jt 



Let Bn{y) = B{Tn+y) — B{Tn). Then conditional on ^^„, Bn{y) is a standard 
Brownian motion. It is obvious that 

dB^^{y) 
T„(t-i) Tn + y 

Hence, conditional on Bn(t), i > 1, is a mean zero Gaussian process with 
covariance function 

/"OO J 

/ _f ^2 = « for t > s > 1. 

It follows that Bn{t), t > 1, is a standard Brownian motion and is independent 
of So, it is independent of J-n because J-n C The proof of the main part 
of the corollary is now completed. 

Now for (pTTl) . from (|2^ it follows that 



dist{Wn{-),anBn{-)) ^ a.s. in D[l,oo), 

where dist{-, •) is a metric in D[l, oo). Note that ct„ is ^„-measurable and — > 
5 a.s., and, conditional on J-n, Sr„_B„(-) and cr„-B'(-) has the same distribution. 
Hence, (|2.7p follows from (j2.8p by noting the following fact that: 



^„(-)|^ A (T a.s. and (iisi((^„(-), 77„(-)) a.s. in£'[l,oo) 

=^ '7n(-)|^„ ^ o^S'(-) a.s. in D[l,oo). 

This fact follows from that for any bounded and uniformly continuous function 

/ : D[l,oo) (-oo,cx)), 



lim sup 



E[/(??n)"/(^„)|-F„] 



limsup |/(?7„) - /(^„)| Y J",J =0 a.s., 



<E 

due to Lemma A. 2 of Crimaldi (2009). 

Finally, (111)) follows from ([276)) by letting (n = ^n(l), and ((2?5|) is a conclu- 
sion of d^HI or dUT]). Aletti, May and Secchi (2009) showed that (^3]) implies 
Zoo having no point masses in (0, 1) by utilizing a metric of the weak convergence 
of probability measures with the limit distribution being absolutely continuous. 
Here we give a straightforward proof. Let f{t) = e~* be the characteristic 
function of a standard normal distribution. Firstly, note that (|2.5p implies that 
for every \/,^^ J^„-measurable event E, 

lim E[e^*v/^(^"-^~)/B|J-„] = f{at)lE a.s. 

n— ^oo 

In fact, if let /„ = E[/£;|J^,i], then /„ Ie a.s.. And hence 

lim E[e'*^(^"-^~)/£;|J-„] = lim E[e'*^(^"-^~)/„| J'„] 
= lim E[e^*v^(^"-^-)|J-„]/„ = /(5i)/£; a.s.. 
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where in the fist equality we use the fact that 

T]n^ a.s. and \rjn\ < M a.s. E[?7„|J"„ 



a.s.. 



This fact is due to Lemma A. 2 of Crimaldi (2009). Next, choosing E — {Z^o = 
p}, p £ (0, 1), yields 

f{^t)lE = lim E[e**^(^"-f)/B|J-„] 

n— j-oo 

lim e'''^^^"-P^E[lE\J'n] = lim e'*^^^"-^)/^ a.s.. 

Hence, \f{at)\lE = Ie a-S.. So = a.s. because \fiat)\ <1 on E. The proof 
is now completed. □. 

From the above proof, we obtain the following corollary. 

Corollary 2.4 Suppose {Yrx,,Yn^n > 1} is a sequence of random variables, 
{a„} is a sequence of constants with a„ — > oo, and is a filtration of a- 
fields such that Yn is 'Sn-measurahle. If for almost every uj, the distribution of 
a,n(Yn — Yoo) conditional on'Sn converges to a non- degenerate distribution, then 
Yrxi has no point masses. 

Remark 2.4 By i2.5\) . it can also be shown that for every event E, 

exp{-itn^^^Z^}lE] 



lim sup lim sup 

|t|-yoo rn-oo 

< lim sup lim sup E 

|t|->-oo ri-s-oo 



exp {itn^''^{Zn - Zac)}lE\j^n] 



= lim sup lim sup f | (icr„) /b | | 

>-oo n— >-oo 

= limsup £[ |/(iCToo) I -^b] =0. 

\t\^oo 

If we denote fE{t) be the characteristic function of the conditional distribution 
of Zoo given E with P{E) > 0. Then the above equality means that 



lim lim sup 

i-oo n— >-oo 



fE{tn 



1/2n 



0. 



Note that for any t^ > \, uq > 1 and \s\ > 2ton(), there exist a real number t 
with \t\ > to and an integer n > uq such that s = tn^/'^ . We conclude that 

lim /i5(s)=0. 

This is related to the Cramer condition. Obviously, if E — {Zoo — p}, p € (0, 1), 
and P{E) > 0, then = 1 which is a contradiction. 



The next corollary is the central limit theorem for the random number of 
draws. 
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Corollary 2.5 Suppose mi = m2 > 0, EUf j, < oo for some p > 2. k = 1,2. 
Then 

yfn {^-^ - 4 h{uj) ■ N{0, 1) stably, (2.9) 

where h{uj) = ^Z^(\ - Z^)^{\ - Z^){2a\ -1) + Z^{2gI - \), and N{0, 1) 
is a standard normal random variable which is independent of Too ■ 

Proof. We need to prove 

\Jn Zo, 

V " 

^ - - ^oo)A^i (0, ai - r 

+ - ^00^00^2 (O, (7^ - 1)) + a . iV3(0, 1) stably, 

where A''fc^O,cr^, ~ 1 j > ^ = 1:2, -/V3(0, 1) are three independent normal random 
variables which are independent of Tr^, ■ Write 

= ^r^i-^'^^(^'.'-/-'--^) , ,^1,2. (2.10) 

Then 

^n,/c = 0(V""^loglogri) a.s. and ^^7— = 1 + A„,fc. 



By the Taylor expansion and (j2.8p . we have that 

Zoo — 



Z„(l - Z„)(A„,i - A„,2) + (Z„ - Zoo) + O 
•^n(l ^ -^ri) / , \ . Zn{l — Z, 



log log n 



n 



+ ?nCn + o(n-l/2) a.S.. (2.11) 

Note that (,„ is a standard normal random variable which is independent of 
Zn, Nn^k and An,k, k = 1,2. Also, ?„ ^ ct a.s., Z„(l - Zn)y^/ y/TT^j 
V%I(1 - Zoo) a.s. and Z„(l - Zn)^/n./ y^N^ Z^oVT^^Z^. The proof is 
completed if we have shown that 

{^/NZiAnS,V^An,2) ^ [Ni{0,<jl~l),N2{0,al~l)^ mixing. (2.12) 

Note that ct^ — 1 = Var([/ijt/mfc) , fc = 1, 2. The above convergence follows from 
Theorem 4.1 of May and Flournoy (2009). □ 

The next corollary tells us that conditional on the conditional distribu- 
tion of y/n{Nn,i/n — Zoo) does not converge. 
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Corollary 2.6 Suppose mi — m2 > 0, EUf f. < oo for some p > 2, k — 1,2. 
Let E he an event that for uj Cz E there is a distribution _Flj for which 



V^{^ - Z^) 4 F^. (2.13) 



Then P{E) = 0. 



Proof Recall (E1T|) . Let ry„ = ^nZ„(l - Z„)(A„,i - A„,2)- Note that 
Zn ~^ Zoo a.s., 

CTnCnl^c-^^ aN{Q, 1) a.s. 

and that Z„, v4„_i and 74„^2 are J>j-measurable. By (12. lip and (|2.13p . there ex- 
ists an event with P(rio) — 1 such that y/nZn{l — Zn){An.i — An,2) converges 
on E nrio. So there exists a random variable rj such that 

r]n{Lo) ri{uj) \fu> G E' n flQ. 

Suppose P{E) > 0. Choose a; such that P{r] > x,E) > 0. Then it follows that 

P{Vn < x,r] > X, E) P{rj < x,r/ > x, E) = 0. 

So P(^„ < > _E) — > 0. On the other hand, according to (|2.12p we have 



\/nZn(l — Zn) I , \ nZn{1 — Zn) , , d , ^ 

?7„ = ^ "i^ ^/l^iAn,!-^ ^l— ^ 7a^A„,2 4 7V(0, 1) mixing, 

where A^(0, 1) is independent of Too- It follows that 

limP(?7„ < x\r] > x,E) = > 0. 

n 

We get a contradiction. The proof is completed. □ 



3 Unequal reinforcement mean case 

In this section, we consider the case of mi ^ TO2- Without loss of generality, we 
assume that < toi < m2. Denote p — mi/m2, and 

V'n = TFF'^ V'oo = lim tpn. 

May and Flournoy (2009) proved that the limit ipoo exists almost surely with 
P(0 < V'oo < oo) = 1 when the reinforcement distributions /ii and /i2 have 
bounded supports. Durham and Yu (1990) proved a similar result as that 

p converges almost surely to a finite limit 7700- 

^n,2 
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It is easily seen that 



'"-2 ; 

nil 



and 



lim — — = Tpoo U.S., lim — — = m2Woo cl-S. 



In a recent manuscript of Zhang ct al (2010), it is proved that the weakest 
condition for P(G < ipao < oo) = 1 is that E[Ui^k log^ C^i.if] < oo, fc = 1, 2, and 
a general multi-color RRU is consider. For the completeness of this paper, we 
will give a simple proof under the assumption of finite (1 + e)-th moments for 
the two-color case. The following is the result. 

Theorem 3.1 Suppose EUf f. < oo for some q > 1, nik > 0, k = 1,2. Then 
the limit V'oo exists almost surely and P(0 < V'oo < oo) = 1 both when nii ~ m2 
and TOi ^ 777.2- 

The following theorem is our main result on the Gaussian process approx- 
imation for ■0„. From the Gaussian approximation we are able to show that 
V'oo has no point masses in (0,oo). And accordingly, all the limits of the se- 
quences {Yr^^i/Yl^}, {l^^.i/n"}, {Y„,i/\Y„\p}, {Af„,i/<,2} and have 
no point masses in [0, oo]. 

Theorem 3.2 Suppose ni2 > mi > 0, EU^ f. < oo, k = 1,2, for some p > 2. 
Denote al = E[{Ui,k/mk)'^], fc= 1,2. Let Sq^ mm{{l~ p)/p,l/2-l/p}. Then 
(possibly in an enlarged probability space) there is standard Brownian motion 
B{y) such that for any Q < 5 < 5^, 

^o.-^„=^r '-^+oin-P^^-^y^) (3.1) 



^ ,-p/2 '^lV^W^ K/V'oo) + ,(,-p(l+^)/2) (3.2) 

m^/' yJnP/iP^ 

where W{x) — —x y^^dB{y) is also a standard Brownian motion. 

Furthermore, the Brownian motion B{y) can be constructed with a filtration 
ofa-fields and a non- decreasing sequence of stopping times {T„} satisfying 

Properties (a) and (c) in Theorem \2.2\ and 

Property (b') T„ = n^'/V'oo + o{nP^^'^^) a.s. for0<6 <So. 

The proof of this theorem will be given in the last section. Next, we state 
several corollaries. The first one is on the law of iterated logarithm and the 
central limit theorem for ijj„. 



Corollary 3.1 Under the conditions in Theorem \3.2\ 



lim sup -j^ a.s. (3.3) 

n^oo V21oglogn jnP'^ 
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and there exists a sequence {Cn} of standard normal random variables for which 
(^n is independent of Tn and 



n^^^^tpn - tpoo) = — — TTT^Cn + o{n a.s. for some e > 0. (3.4) 



p/2 



Hence 



-(^n-V-oo)!^ -A7v(0,^i^VooH) a.s. (3.5) 
and "000 has no point masses in (0,oo). 

Proof. p.3p follows from p.2p and the law of iterated logarithm of the 
Brownian motion. p.4p can be proved in the same way as proving Corollary 
O □ 



Corollary 3.2 Under the conditions in Theorem \3.S\ 



- Voo) A N{0, 1) • Jvoo{2ctI - 1) stably (3.6) 

and 



'nP/\^ ~ 7/oo) ^ Af(0, 1) • .Jiioo{2ai - 1) «/p<2/3 



(3.7) 



n^-"{l-^)^Voo a.s. (3.8) 
Proof For dM]), let ^„,fc be defined as in ([2^0]) . Then 

^„,2 - 0(^iV-iloglog7V„,2) = OiVn-noglogn) = o(n-''/2-^) a.s., 

- 0(Y/7V-jloglogA^„,i) = ©(V"-" log log n) a.s.. 
Note that 7700 — ipoo'm'2/mi. It follows that 
Nn.i , , mP (l + A„,2)P 

= - ^00 + V-n — (1 - ^n.l) + o(n-''/2-^) 

TOl mi V 

The proof of p.6p is completed by noting that ipn'nii / mf^ — > 7700 a-S., Nns/n^ — >■ 

mixing, and Cn is a standard normal random 
variable which is independent of Tn- 
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For p.7l) . it is sufficient to note that 



Nnl /^nl\2\ 

n \ n / / 



<2 '^K. 



K2 



3.81) is obvious because 



1 — — — ^ a.s. 

n n n 

The proof is now completed. □. 

Finally, we give the functional central limit theorem. 

Corollary 3.3 Define 

W„(t)=n''/V(V'[„t]-i^oo),i>0. 

Then 

Wn{t)^aiy^B'{tP) stably, (3.9) 

in the Skorokhod Topological space D(0,c»), where B'{t) is a standard Brownian 
motion which is independent of !Fao . In particular, 

hm P { ^^-0<^<rJ^~^n) ^ > 0, (3,10) 

Proof. The proof of p.9p is similar to that of (|2.3p by noting that rjao = 
ipoomi/m2. For (|3.10p . it is sufficient to see that 

max ^"^ = sup (W^it) - fW^il)) 

0<l<n nPl^ 0<t<l 

and iV„ 1 ~ rjoanP a.s.. □ 



4 Concluding Remark 

We approximated Z„— Zoo and ipn—i^oo by a kind of Gaussian process y~^dB{y), 
which is a tail stochastic integral with respective to a Brownian motion, with 
time t stopping at a random variable niJ or n/^/ioo, where iJ^ = (l+V''oo)(o'i/'0oo + 
trl). But this does not mean that J,^^ y^^dB{y) and /„y^ y^^dB{y) are Gaus- 
sian random variables and their distributions are unknown because the mixing 
distribution of V'oo is unknown. For deriving the asymptotic distributions, the 
approximations (j2.6p and p.4p seem more powerful than (12. ip and p. II) because 
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the process for approximation is independent of other random variables consid- 
ered. (|2.ip and (|3.1|) are helpful for establishing the strong convergence such as 
the law of the iterated logarithm. 

It is of interest to find the distribution of ipoo- In the case that lo.fc and Um,k, 
k = 1,2, are all integers, in a recent manuscript of Zhang, et al (2010) it is proved 
that the distribution of 'i/'oo is absolutely continuous and is determined by lo.ii 
Fo,2 and the distributions of Ui^i and if E[J7i,fc log"^ Ui^k] < 00, k = 1,2. In 
the general case, the distribution of ipoo is still a open problem. In our Corollaries 
l2.3l and [5TTl by applying the Gaussian approximation and a clever idea of Aletti, 
May and Secchi (2009) we show that V'oo has no point masses in [0,oo] under 
the assumption of finite (2 + e)-th moments. The next step is to show that the 
distribution is absolutely continuous. Unfortunately, as discussed in Aletti, May 
and Secchi (2009), the almost-sure conditional central limit theorems (12. 5|) and 
p. 51) are not enough to prove the absolute continuity. Our method and that of 
Aletti, May and Secchi (2009) depends on the martingale approach, which is 
not a very powerful tool to derive the limit distribution which is not normal. 
To find the exact distribution of V'oo needs new methods. 

For a special case that P{Um,k = 0) = p and P{Um,k — ct) — 1 ~P with a > 
and < p < 1, in Aletti, May and Secchi (2007) it is shown that the distribution 
of Zoo is a beta distribution, and hence the probability that ipoo — Z^o/ (1 — Zoo) 
falls into any subset of [0, 00] will not be zero if the Lebesgue measure of this 
subset is positive. So, it is also of interest to prove in the general case that the 
probability of ipoo falling into any nonempty subinterval of (0,cxi) will not be 
zero. 

Finally, this paper only consider the two-color urn model. In the manuscript 
of Zhang, et al (2010), the asymptotic properties for a multi-color reinforced urn 
model are studied. It is expected to approximate the urn components after being 
suitably normalized by a multi-dimensional Gaussian process. The Skorokhod 
embedding method used in this paper does not work for the multi-dimension 
case. Though strong approximations for multi-dimensional martingales can be 
found in literature, for example, Monrad and Philipp (1991), Eberlein (1986) 
and Zhang (2004), the martingales concerning to the reinforced urn model usu- 
ally do not satisfied a condition that the asymptotic conditional variability is 
J-fc-measurable for some fixed k (c.f., (jA.5[) ). which is needed in the approxima- 
tion theorems for multi-dimensional martingales. A new approach is needed for 
approximating the multi-color reinforced urn models. 

A Proof of the main results 

Recall \Yn\ = Yn.i+Yn,2, P{XnM = l|-F„_i) = Yn-i.k / \Yn-i\. Wc first prove 
Theorems O and O 

Proofs of Theorems \2.1\ and \3.1\ Without loss of generality, assume 1 < 
q < 2. It is obvious that Theorem 12.11 follows from Theorem 13.11 with Zoo = 
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■0oo/(l + ^/-oo). For Theorem Ol let 

1 



Qr. 



— iogr„.i iogy„.2. 

TOl 7712 



Then Q„ — — log V'n- So, it is sufficient to show that converges almost sure 



to a finite limit. Write 

AQ„ ~Qn — Qn-l 
1 



— ^n.llog 1 
7711 



y Un,i/mi 

^ri,l— ;7 ^n,2 



Un.l 



Yn-1,1 

Un,2/m2 



— Xn.2 log I 1 
7712 



Un 



Yn-1,: 



Yr, 



-X„i/fi^U^X„2/f^) 



777 

(1) , A^(2 



(A.l) 



where /(x) = x— log(l+a;) satisfying < f{x) < a;' for a; > 0. We need to prove 
the almost sure convergence of the random series X)n=i ^Qn- We first consider 
the terms in the second bracket above. Denote — cr(-7>i, -^^n+i,!, -'fn+1,2)- 
Note that K„jt « Nn^k a.s. by Lemma A. 4 of Hu and Zhang (2004), and then 
l^il ~ A^n,i + Nn^2 = n a.s.. We have that 



r TT 

n— 1 

oo 



0-1 "'A': 



< 



^ 1 



n=l 









■04-1 



{ (1 + iV„,fe)'? 



dx < C 



dx 



(1 + 



< oo a.s. 



Hence J2'^=i ^Qn'' converges almost surely. For {AQn ^}, it is easily seen that 
it is a sequence of martingale differences with respect to the ct- filtration 
and 

So, it is sufficient to show that X^^i l/d^-il^^i fc) < °^ a.s., fc = 1,2. The 
proof will be completed if we have proven that Yn^k > n'^ for some positive e. 
Now, it is obvious that 



^(iog7i)-'E[|Agwnj-„_i] < 

n=l 

which implies that 



^^JK_l|(l0g77) 

Qr. 



< OO a.s. 



AQ^^' — 7^ a.s., and hence 7-^^^^^ > a.s. 

I0g77^ " 



1 = 1 



log 77 
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On the event > ^71,2}, we have that 

log y„,2 = - ■m2Qn H log I y„ I H log — - 

mi mi \Yn\ 

7712 "^2 1 

> ~ m2Q„ H log I r„| H log -. 

mi mi 2 

On the event < ^,2}, we have y„,2 > 11^1/2. Note that \Yn\ ~ n. It 

follows that 

. r logF„,2 ^ "^2 . , 
ummt — > A 1 a.s. 

n^ao logn TOi 



Similarly, we have that 



. f logy«,i ^ "^1 . 1 
limmt — > A 1 a.s. 

n-i-oo log n 1712 



The proof of Theorems 12.11 and 13 . 1 1 is now completed. □. 

Proof of Theorem[EE Let f{x) = e'"i^/(l + e™!^). Then /(Q„) = Z„ and 
f'iQoo) — '7iiZoo(l — Zoo)- According to the Taylor expansion, it is sufficient 
to show that B{t) and T„ can be constructed such that 

Qoo-Q« = — / — + o{\n) a.s.. A.2 

mi J^H X 

Recall mi = m2. It is easily shown that — ?> toi a.s.. According to 
Theorem 12.11 Z^o G (0,1) a.s., which implies that y„ fe w n a.s., k = 1,2. So, 
for AqL^^ in we have 

which implies that Y17Ln+i I^QP^I = o(A„) a.s.. 

For AQi^\ we use the truncation method. Let Un,k — Un.k/fT^k Un,k — 

Un,kI{Un,k < n^/P}, CT^ = ^U'^n,k^ '^n,k = E[C/„,fe], /c = 1, 2, TO„ = m„,i-m„,2, 

and 



AM^i) = mm XnSTT^ - X 



ri,2 



ri,2; 



AM„ = AM(i) - E[AM(i)|^„_i] = AAf(i) - 
Then {AAf„, J>t} is a sequence of martingale differences. Note that 

oo 

^ P{Un^k/mk > ni/P) < E(C/i,fc/mfe)P < oo, = 1, 2. 

n=l 
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From the Borel-Cantelli lemma, it follows that 

P(AQ(,i) ^ ^AM„ i.o.) = 0. 
min 

Also, 

1 CO I — I 



i— n+1 

oo 2 2 

/=ri+l fc=l k=l 

Hence, we conclude that 

oo oo ^ 



mi I 

l=n+l ;=n+l 



+ o(A„) a.s. 



For the martingale M„ = X^ILi ^-^/j we have 



.-rT2 



E[|AM„hJ-„_i] <^ ("^^^VeC/:,, < Ciu;)j2^ut,k- (A.4) 
fc=i V^"-i,fc/ fc^i 

By the Skorokhod embedding theorem (c.f.. Theorem A.l of Hall and Heyde 
(1980, page 269)), (possibly in an enlarged probability space) there is a standard 
motion B{x) with a filtration and a sequence of nonnegative stopping 

times Ti , T2 , • • • with the following properties 

(i) M„ = S(T„), where r„ = Y.ti 

(ii) J-„ C r„ is^f„ measurable, E[r„|^„_i] - E[(AM„)2| J-„_i], E[t;;|^„_i] < 

aE[(AM„)2-|J-„_i] for any r > 1; 

(iii) Conditional on i?(T„ + x) — B{Tn), a; > 0, is also a standard Brownian 

motion. 

Now, we verify that the Brownian motion B(x) and the stopping time T„ 
are desirable for Property (b) and (jA.2p . At first, we assume the following 
approximation for the conditional variance. 

E[(AM„))2|J-„_i] = Hiuj) + oin^^P-^) a.s. (A.5) 
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From fjA.5p it follows that 

71 

J2 E[t„|^.-i] = nHiio) + o{n^/P) a.s 
On the other hand, by (ii) and (jA.4[) we have that 

X]a:=1 n,k 



n—1 71—1 

By the law of large numbers of martingale, it follows that 



<C^E[(C/i,,/m,)f] <oo. 
fc=i 



^(T,-Eh|^^,.i]) = o(n2/f) a.s.. 



Hence 



Property (b) is verified. Then it follows from the path properties of a Browian 
motion (c.f., Theorem 1.2.1 of Csorgo and Revesz (1981)) that 



B[Tn) - B{nH) = o 



+ ) = o(nA„) a.s. 



So 



^(E^-(-)-^)+El(yiT)«(-0.o(A. 

\l—n / /— n 



?71i 

1 

771 1 

H [°° dB{x) 



B(xH) , B(nH) 

5 — dx 

x'^ n 

B(x) , BinH) 
— T^dx 



+ o(A„) a.s.. 



o(A„) 
- o(A„) 



Finally, we verify (jA.Sp . Note that j,] ^ cr^, |l^„|/n toi a.s., fc = 1, 2 
and Z„ — > Zoo a.s.. ()A.5P is obvious for p = 2 by (jA.Sp . 

For 2 < p < 4, we still have Corollary \2.1\ due to the approximation for 
the case of p = 2. Hence 



Z„ - Zoo = 0{^n-Hog\ogn) = o(n2/p-i) 
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On the other hand, 



min 



rriin 



--0{y/n-^\og\ogn) = o{n^''P-^)) a.. 



and 



fc=i 



al - E[ulj = E[(C/i,fe/mfe)2/{;7i,fc/TOfc > n'/P}] = o(n2/P-i), fc 1,2. 
(jA.SP follows by (|A.3P . The proof is now completed. □ 

Proof of TheoremU^ As before, denote |y„| = Yn^i+Yn^2, Qn = logKa- 
logF„^2- According to the Taylor expansion, it is sufBcient to show that B{t) 
and T„ can be constructed such that 



QoQ Qn 



CTl 



dB{y) 



+ 0(71 



Recall (|A.1|) and note that 1^-1,2 ^ m2n, ~ i'ooY^2 ^ i'oo{m2n)P a.s. 

It can be show that for < (5i < 1/2 and (5i < (1 — p)/p, 



;p(l+5l)/2 

Tip' 



< 00. 



which implies Ei^„+i^i,fe/(C^i,fc/>l-i,fe) = o(n-''(i+*i)/2) a.s.. A: = 1,2. Also, 

1 

Yi-1,2 ]Yi~T\- 



for the martingale differences X/,2-^rT^ ~ rvrTT' have 



^(^p(1+5,)/2)2e[(x;.2^ 



Uu2/m2 1 



l^-il 



- ;p(i+5i) 
< — j3 — < 00 a.s., 



«2 



which implies YZ^+i {Xi.2^rrr^ " ^r^) = o{n-p'-^+^^^/^) a.s.. Similarly, 
we can show that J^lZn+i ~ ITTTt) ~ o{n^P^^ logn) a.s. It follows 



that 



Qoo - Q„ = f: (^U^^ - ^) + oin-^^'+'^y^) a.s. (A.7) 



;=n+i 



and 

Define 



Qoo-Qn = o{n pI"^ log n) a.s. 

^pf?I^{m2n)p/^nP/^ / C/„,i/mi 
Aik/„, = I A„ 1 



(A.8) 
(A.9) 
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Then 



irr^A»/r\2nr 1 pm2{m2n)PnP pm2{m2n)PnP 



Next, we first show that 

E[(AAf„)2|j;„_i] = ^nP-\l + o(n-''*0) a.s. (A.IO) 

From (jA.8[) and the Taylor expansion, we conclude that 

^'^ ijjoo — o{n''P^'^ logn) a.s. 



(Yn,2)P 

On the other hand, 

2 



m2n mi n ^-^ n 

k—l 

=1 - ^M^^ + o{n-p/^-^ \ogn) + 0(n-i/2(loglogn)i/2) 
mi n 

=1 - 0{nP-'^) + o{n'P^') = 1 + o{n-P^') a.s. 



It follows that 

1^1 Yn,2 Yn.l , N •, ^n,l , , -pi5i \ 

— 1 = 1 + o(n P ^ ) and r -^oo — o[n p ^ ) a.s. 

771211 771271 7712^1 (7n27l)P 

(jA.lOp is verified. From (|A.10I) . it follows that 

n p 

E[(AAf„)2|J-„_i] = ^(1 + o{7i-p'')) a.s. 

1=1 

On the other hand, for < (^2 < 1/2 - 1/p, 

E[|AM„|P|J-„_i] 7iPP y„_i,i 



(„p(i-A-2))p/2 - „p(i-52)p/2 yP_^ 

1 1 

— „p(l-(52)p/2 „1- 



j^p(l-(52)p/2 ^1-p 
n=l 



So, similarly as in the proof of Theorem 12.21 by the Skorokhod embedding 
theorem, the standard motion B(x), the filtration {^^n} and the stopping times 
{Tn} can be constructed such that M„ = B{Tn) and 

T„ = E[(AM02|J-/-i] + o{77p^^-'-^^) = -^n" + o(n''(i-«i^«^)). 
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Denote Sq — min{(l — p)/p, 1/2 — It is remained to verify p. II) . By the 

Properties (b') and the path properties of a Brownian motion, we have for any 
< 5 < 60, 

M„ - B(n7^oo) = oK^i-^)/^) a.s. 



Hence 



IP ^\IP (l + l)Pj ' nP 

l—n 

1 ^ on./.,. ^ SK/V-oo) 



E 



{i + iyj nP 

^(;p(l-5)/2) ^(^p(l-5)/2) 



l—n 

„p ^ ' 

ir- ^,,_BinP/^^)^ 

Woo Jn"/^^ X nP 
1 1"°° dB{x) -p(1+5)/2n 

' ax + o(n P^ ^ " j a.s. 



(|A.6p is now proved by noting that (|A.7|) . (|A.8p and pm2 = toi- And hence 
dm is verified. □ 

Remark A.l Using the truncation method as in the proof of Theorem \2.2[ we 
can proved that \3. l\l remains true under the assumption of only finite second 
moments if n~P^^~^^^/'^ is replaced by n~''/^(loglogn)^/^. This implies that the 
law of iterated logarithm \3. S\l remains true when EU^ < 00, k — 1,2. 
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